Pipeline architectures are implemented to process data and include processor chains (also commonly referred to as “graphs”) that pass data and commands through an ordered chain of data processors which each perform a designated data processing function. As such, data inputs to a pipeline architecture enter at one or more defined processing point(s) (e.g., a first data processor), and data outputs from the pipeline architecture are expected at one or more defined processing point(s) (e.g., an N-data processor in the processor chain).
Typically, the data links that connect the data processors of a processor chain are used to communicate both the data to be processed and command traffic, such as data instructions that act upon a data processor to modify an internal state or an operation that the data processor performs to process the data. The command traffic is communicated through a processor chain from an input to the output of each data processor in the chain such that each data processor affected by a command message will have the chance to operate on it.
This simple pipeline architecture design can produce unexpected data outputs and data processing errors when there are data processors that have multiple outputs and/or multiple inputs. A data processor within a processor chain may split a data input and then pass the data sub-streams through different data processors of different processor chains which are then re-unified at a later processing point in the chain.
These data processing splits can cause command traffic management issues that may result in unexpected data outputs and data processing errors. For example, the data processor that receives the data sub-streams and merges together the data from the two different processor chains may receive as many copies of each command as there are processor chains which input into it, thus causing errors in the data processing for non-idempotent commands. Another potential issue is that some data processors in a-processor chain may act on a command message before all of the preceding processors that provide an input to them have completed acting on the command message which may also cause unexpected data outputs and data processing errors.